On the plausibility of simplified acoustic room representations for listener translation in dynamic binaural auralizations

Diese Doktorarbeit untersucht die Wahrnehmung vereinfachter akustischer Raumrepräsentationen in positionsdynamischer Binauralwiedergabe für die Hörertranslation. Die dynamische Binauralsynthese ist eine Audiowiedergabemethode zur Erzeugung räumlicher auditiver Illusionen über Kopfhörer für virtuelle, erweiterte und gemischte Realität (VR/AR/MR). Dabei ist es nun eine typische Anforderung, immersive Inhalte in sechs Freiheitsgraden (6DOF) zu erkunden. Dynamische binaurale Schallfeldimitationen mit hoher physikalischer Genauigkeit zu realisieren, ist meist mit sehr hohem Rechenaufwand verbunden. Frühere psychoakustische Studien weisen jedoch darauf hin, dass Menschen eine begrenzte Empfindlichkeit gegenüber den Details des Schallfelds haben, insbesondere im späten Nachhall. Dies birgt das Potential physikalischer Vereinfachungen bei der positionsdynamischen Auralisation von Räumen. Beispielsweise wurden Konzepte vorgeschlagen, die auf der perzeptiven Mixing Time oder der Hörbarkeitsschwelle von frühen Reflexionen basieren, für welche jedoch eine gründliche psychoakustische Bewertung noch aussteht. Zunächst wurde ein Aufbau zur positionsdynamischen Raumauralisation implementiert und evaluiert. Daran untersucht die Arbeit wesentliche Systemparameter wie die erforderliche räumliche Auflösung eines Positionsrasters für die dynamische Anpassung. Da allgemein etablierte Testmethoden zur wahrnehmungsbezogenen Bewertung von räumlichen auditiven Illusionen unter Berücksichtigung interaktiver Hörertranslation fehlten, untersucht die Arbeit verschiedene Ansätze zur Messung der Plausibilität. Auf dieser Grundlage werden physikalische Vereinfachungen im Verlauf des Schallfeldes in positionsdynamischen binauralen Auralisationen der Raumakustik untersucht. Für die Hauptexperimente wurden binaurale Raumimpulsantworten (BRIRs) entlang einer Linie für die Hörertranslation in einem eher trockenen Hörlabor und einem halligen Seminarraum ähnlicher Größe gemessen. Die erstellten Datensätze enthalten Szenarien von Hörerbewegungen auf eine virtuelle Schallquelle zu, daran vorbei, davon weg oder dahinter. Darüber hinaus betrachten die Untersuchungen zwei Extremfälle der Quellenorientierung, um die Auswirkungen einer Variation der Schallquellenrichtcharakteristik zu berücksichtigen. Die BRIR-Sätze werden systematisch bearbeitet und vereinfacht, um die Auswirkungen auf die Wahrnehmung zu bewerten. Insbesondere das Konzept der perzeptiven Mixing Time und manipulierte räumlich-zeitliche Muster früher Reflexionen dienten als Testfälle in den psychoakustischen Studien. Die Ergebnisse zeigen ein hohes Potential für Vereinfachungen, unterstreichen aber auch die Relevanz der genauen Imitation prominenter früher Reflexionen. Die Ergebnisse bestätigen auch das Konzept der wahrnehmungsbezogenen Mixing Time für die betrachteten Fälle der positionsdynamischen binauralen Wiedergabe. Die Beobachtungen verdeutlichen, dass gängige Testszenarien für Auralisierungen, Interpolation und Extrapolation nicht kritisch genug sind, um allgemeine Schlussfolgerungen über die Eignung der getesteten Rendering-Ansätze zu ziehen. Die Arbeit zeigt Lösungsansätze auf.This thesis investigates the effect of simplified acoustic room representations in position-dynamic binaural audio for listener translation. Dynamic binaural synthesis is an audio reproduction method to create spatial auditory illusions over headphones for virtual, augmented, and mixed reality (AR/VR/MR). It has become a typical demand to explore immersive content in six degrees of freedom (6DOF). Realizing dynamic binaural sound field imitations with high physical accuracy requires high computational effort. However, previous psychoacoustic research indicates that humans have limited sensitivity to the details of the sound field. This fact bears the potential to simplify the physics in position-dynamic room auralizations. For example, concepts based on the perceptual mixing time or the audibility threshold of early reflections have been proposed. This thesis investigates the effect of simplified acoustic room representations in position-dynamic binaural audio for listener translation. First, a setup for position dynamic binaural room auralization was implemented and evaluated. Essential system parameters like the required position grid resolution for the audio reproduction were examined. Due to the lack of generally established test methods for the perceptual evaluation of spatial auditory illusions considering interactive listener translation, this thesis explores different approaches for measuring plausibility. Based on this foundation, this work examines physical impairments and simplifications in the progress of the sound field in position dynamic binaural auralizations of room acoustics. For the main experiments, sets of binaural room impulse responses (BRIRs) were measured along a line for listener translation in a relatively dry listening laboratory and a reverberant seminar room of similar size. These sets include scenarios of walking towards a virtual sound source, past it, away from it, or behind it. The consideration of two extreme cases of source orientation took into account the effects of variations in directivity. The BRIR sets were systematically impaired and simplified to evaluate the perceptual effects. Especially the concept of the perceptual mixing time and manipulated spatiotemporal patterns of early reflections served as test cases. The results reveal a high potential for simplification but also underline the relevance of accurately imitating prominent early reflections. The findings confirm the concept of the perceptual mixing time for the considered cases of position-dynamic binaural audio. The observations highlight that common test scenarios for dynamic binaural rendering approaches are not sufficiently critical to draw general conclusions about their suitability. This thesis proposes strategies to solve this

Data set: BRIRs for position-dynamic binaural synthesis measured in two rooms

Binaural room impulse responses were measured with a KEMAR 45BA head-and-torso-simulator. For the first data set, it was placed at different positions located on a line with a length of 2m in a 25 cm positional resolution and an azimuth resolution of 4 . Two source positions were considered in the setup, one in front of the line, one at the side. The same arrangement of source and receiver positions was realized in two different rooms, a quite dry listening laboratory and a quite reverberant seminar room. For the second data set, BRIRs and omni-directional RIRs were measured for a translation line with a length of 7.5m through the given seminar room. The data sets are valuable for realizing, testing and studying dynamic binaural walk-through scenarios in the two different rooms

Real-time Estimation of Reverberation Time for Selection of suitable binaural room impulse responses

The aim of auditory augmented reality is to create a highly immersive and plausible auditory illusion combining virtual audio objects and scenarios with the real acoustic surrounding. For this use case it is necessary to estimate the acoustics of the current room. A mismatch between real and simulated acoustics will easily be detected by the listener and will probably lead to In-head localization or an unrealistic acoustic envelopment of the virtual sound sources. This publication investigates State-of-the-Art algorithms for blind reverberation time estimation which are commonly used for speech enhancement algorithms or speech dereverberation and applies them to binaural ear signals. The outcome of these algorithms can be used to select the most appropriate room out of a room database for example. A room database could include pre-measured or simulated binaural room impulse responses which could directly be used to realize a binaural reproduction. First results show promising results combined with low computational effort. Further strategies for enhancing the used method are proposed in order to create a more precise reverberation time estimation

Binaural room impulse responses: Same listener-source-setup at different positions in the room

To study the perception of room acoustics in dependency of the position in the room, measurements with KEMAR head-and-torso-simulator were conducted. The dummy head was placed at 5 different positions in a small conference room (10.3mx5.8mx3.1m, RT=0.65s). The source, a loudspeaker Genelec 1030A, was always positioned in the same relation to the listening position. BRIRs were measured with an azimuth-resolution of 5° from 0°-360°. This data allows a psychoacoustical comparison of the room acoustical properties at different positions in the room

Creation of auditory augmented reality using a position-dynamic binaural synthesis system - technical components, psychoacoustic needs, and perceptual evaluation

For a spatial audio reproduction in the context of augmented reality, a position-dynamic binaural synthesis system can be used to synthesize the ear signals for a moving listener. The goal is the fusion of the auditory perception of the virtual audio objects with the real listening environment. Such a system has several components, each of which help to enable a plausible auditory simulation. For each possible position of the listener in the room, a set of binaural room impulse responses (BRIRs) congruent with the expected auditory environment is required to avoid room divergence effects. Adequate and efficient approaches are methods to synthesize new BRIRs using very few measurements of the listening room. The required spatial resolution of the BRIR positions can be estimated by spatial auditory perception thresholds. Retrieving and processing the tracking data of the listener’s head-pose and position as well as convolving BRIRs with an audio signal needs to be done in real-time. This contribution presents work done by the authors including several technical components of such a system in detail. It shows how the single components are affected by psychoacoustics. Furthermore, the paper also discusses the perceptive effect by means of listening tests demonstrating the appropriateness of the approaches

Augmented/Mixed Reality Audio for Hearables: Sensing, control, and rendering

Publisher Copyright: © 2022 IEEE.Augmented or mixed reality (AR/MR) is emerging as one of the key technologies in the future of computing. Audio cues are critical for maintaining a high degree of realism, social connection, and spatial awareness for various AR/MR applications, such as education and training, gaming, remote work, and virtual social gatherings to transport the user to an alternate world called the metaverse. Motivated by a wide variety of AR/MR listening experiences delivered over hearables, this article systematically reviews the integration of fundamental and advanced signal processing techniques for AR/MR audio to equip researchers and engineers in the signal processing community for the next wave of AR/MR.Peer reviewe